Skip to content

refactor(backups): drop node-7z and 7zip-bin by moving backups to gzipped tar - #274

Merged
Pixnop merged 10 commits into
devfrom
refactor/drop-seven-zip
Aug 31, 2026
Merged

refactor(backups): drop node-7z and 7zip-bin by moving backups to gzipped tar#274
Pixnop merged 10 commits into
devfrom
refactor/drop-seven-zip

Conversation

@Pixnop

@Pixnop Pixnop commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Summary

The launcher's own backups were the last thing that needed a 7-Zip process, so this moves them off zip and takes the whole 7-Zip dependency chain out with them. New backups are written as gzipped tar through the tar package the game archives are already read with, which means the writing happens in process instead of by spawning a binary and reading what it prints back. The compressionLevel in the config keeps its meaning exactly: zlib's gzip takes the same 0 to 9 scale, the number reaches the writer untouched, and level 0 still stores rather than deflates. Progress reporting keeps the shape the worker protocol expects, a leading 0 from the handler, deduplicated and monotonic figures from the writer capped a point short of the end, and exactly one terminal 100.

Every backup a player already has is a zip, and those have to keep restoring forever, so the zip reader stays while the zip writer goes. yauzl already reads mod archives and already reads a zip's table of contents before extraction, so the restore path gets a yauzl unpacking branch alongside the existing tar one. Two formats now reach the launcher and no others, and validateArchive refuses anything else by name rather than handing it to a reader that would have to guess. One thing had to change along with the format: the single-wrapping-folder flattening used to be inferred from the file extension, which only worked while backups were the zips and game builds were the tar.gz files. Now that both are gzipped tar, the flattening is asked for by the caller instead. The game version install asks for it, the backup restore does not, and a backup whose only entry happens to be a folder is no longer at risk of being unpacked one level too shallow.

Deleted

Three dependencies (node-7z, 7zip-bin, @types/node-7z), which took 76 lines out of the lockfile and six 7za binaries out of the package. The asarUnpack entry that shipped them, the executable-bit repair in scripts/fix-native-deps.js (21 lines plus the paragraph explaining it: the script itself stays, because its other half downloads the Electron binary that a plain npm ci no longer fetches, which has nothing to do with 7-Zip), the 7-Zip spawn in the extraction worker (19 lines), and the hand-written parse of 7z l -slt listing text in archiveValidation.ts (74 lines). The audit's claim that the -slt parse was unreachable holds up: EXTRACT_ON_PATH has exactly two callers, the game version install, which passes a .tar.gz on Linux and macOS and routes Windows to the installer reader instead, and the backup restore, which passes a .zip. Neither could ever reach the third branch.

Size

A Linux --dir build goes from 367,178,968 bytes to 357,180,720, so 9,998,248 bytes smaller, around 9.5 MiB or 2.7 percent. Most of that is the 9,191,070 bytes of 7zip-bin that used to sit in app.asar.unpacked; the rest is node-7z and its own dependencies inside the asar. Both numbers come from npm run build:unpack on the same machine, measured with du -sb.

Testing

Everything below ran on Linux (Manjaro, kernel 6.18.45, Node 22). CI runs the same suite on ubuntu only, per #267, so nothing here has been exercised on Windows or macOS. The parts most likely to behave differently there are the path comparisons in the zip writer's destination check, which is why that check is unit tested against Windows separators and drive letters directly rather than only through a real archive.

npm ci clean. npm run typecheck, npm run lint:ci (0 errors, 15 warnings, all pre-existing) and npm run format:check all pass. npm run test:coverage is 137 files, 1643 passing and 2 skipped, at 92.46 percent statements, 89.83 branches, 92.07 functions and 93.89 lines, every floor in vitest.config.ts clear. npm run build:unpack succeeds and the output contains no 7-Zip binary, checked by searching the packaged tree for anything named after it.

The round trip is covered both ways. A fixture installation is compressed and restored, and the restored tree is compared file by file against the original, including a zero byte file and a nested folder. A committed zip fixture shaped like the backups the launcher used to write is restored and compared the same way, so the legacy path is held by an archive rather than by a mock. An empty installation round trips to an empty folder rather than to a failed backup.

The path safety on restore is pinned at both gates. A second committed fixture carries three entry names that point outside the folder they would be unpacked into: one climbing with .., one absolute, and one naming a Windows drive. The archive is refused before the output folder is even created, and the test asserts that none of the three landed anywhere. The writer's own resolved-path check is unit tested directly, since yauzl's own name validation refuses these archives first and would otherwise hide it. The tar path gets the same treatment with a hostile archive built in the test.

Pruning, the isRestoring and isDeleting guards, and the backup adapter's verdicts are untouched; only the file extension in their fixtures changed.

Four deliberate breakages, each confirmed to fail tests: removing the legacy zip branch from validateArchive fails 8 tests, removing the escape check from the zip writer fails its unit test, removing the terminal 100 fails 2 compression tests and 2 extraction tests, and flipping the unwrap default to true fails the test that says a backup is never flattened.

Related issues

Addresses the cut proposed in #222, taking the option the issue itself recommended: new backups switch to gzipped tar, old zips stay readable.

Pixnop added 3 commits August 29, 2026 00:10
The backup writer was the only thing left in the launcher that needed a
7-Zip process. It now goes through the tar package the game archives are
already read with, so a backup is written in process rather than by
spawning a binary and parsing what it prints.

The compressionLevel the config carries keeps its meaning: zlib's gzip
takes the same 0 to 9 scale, so the number reaches the writer unchanged
and level 0 still stores rather than deflates.

Progress reporting keeps the shape the worker protocol expects. The
safety walk over the source tree now also totals the bytes it sees, and
each entry written moves the figure, capped a point short of the single
terminal 100 the caller emits at the end.

The tests drive real archives instead of a stand-in for the 7-Zip call,
which is what makes the compression level and the archive's own shape
assertable rather than taken on trust.
Every backup made up to now is a zip, and those have to keep restoring
for as long as players still hold them. yauzl already reads mod archives
and already reads a zip's table of contents before extraction, so the
restore gets a yauzl unpacking path and the zip writer is what goes away.

Two formats reach the launcher now and no others. validateArchive routes
gzipped tar to the tar reader and zip to yauzl, and refuses anything else
by name rather than handing it to a reader that would have to guess. The
hand-written parse of 7-Zip's -slt listing text, which nothing could
reach any more, goes with it.

The single-wrapping-folder flattening moves from being inferred from the
file extension to being asked for. It was only ever meant for the Linux
game archives, and now that backups are gzipped tar too, inferring it
would flatten the restore of any installation whose only entry happens to
be a folder.

Two committed fixtures back this: a zip backup shaped like the ones the
launcher used to write, restored and compared byte for byte, and one
whose entry names climb out with "..", name a drive, and give an absolute
path. Nothing lands outside the destination for any of them.
Nothing spawns 7-Zip any more, so the three packages and the six 7za
binaries they ship go. That takes the bundled binaries out of
app.asar.unpacked, takes the asarUnpack entry that put them there out of
the builder config, and takes the executable-bit repair out of the
postinstall script.

The script itself stays: its other half downloads the Electron binary
that a plain npm ci no longer fetches, which has nothing to do with
7-Zip.
@Pixnop
Pixnop requested a review from Zaldaryon August 28, 2026 22:12
Pixnop added a commit that referenced this pull request Aug 28, 2026
None of them turned out to be a Windows bug in the launcher. The twelve
EXECUTE_GAME failures all came from one fixture assumption: every test
in gameHandlers.test.ts writes a game binary called "Vintagestory", and
buildGameLaunchPlan only ever looks for "Vintagestory.exe" on Windows.
So the folder held no game, the handler answered no-executable, and
every outcome those tests were written for went unreached, adoption
included. Renaming the file by platform in one place restores all
twelve, and the two that looked like they diverged into session
adoption with mismatched uids were the same cascade one level further
down: with no launch there is no session write, so nothing was there to
adopt. Reproduced on Linux by pointing the same helper at a name the
launcher does not know, which fails all twelve with the exact Windows
messages, mismatched uids and all.

Three tests in the same file cannot work on Windows whatever the binary
is called. Two make a write fail by taking write permission off the
installation folder and one makes a folder unlistable with chmod 0o000;
NTFS has no such mode bits, so the write lands and the folder lists.
They skip there with the reason on them.

The three CHANGE_PERMS failures share one cause too: the handler
returns false on anything that is not Linux before it looks at its
arguments, which is right, since POSIX mode bits are the only thing it
has to apply. So the two validation tests get no throw to catch and the
worker test waits for a worker that is never started. Also reproduced
on Linux, by making that early return fire here.

Of the two in extraction.test.ts, one asked the filesystem whether
"vintagestory" exists to prove the wrapping folder was flattened away,
which on a case-insensitive filesystem answers about the "Vintagestory"
file sitting next to it. The full listing on the line above already
says it, and says it better, since an extra folder could not hide from
it either; breaking the flattening still fails the test with that line
gone. The other spends two 7-Zip processes on a 2000 file archive and
runs past the five second default on a Windows runner. Nothing in the
coalescing it covers is platform-specific, and #274 replaces the test
with a yauzl one that spawns nothing, so it skips on Windows for now.

@Zaldaryon Zaldaryon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on one point. The refactor itself is clean: the flatten flip is behaviour preserving, the removed 7z -slt branch really was unreachable, the dependency and config removals are complete, and the two validation gates plus the fresh temp dir hold up against zip slip on both readers. Local gates pass here too (typecheck, lint:ci at 0 errors and the same 15 pre-existing warnings, format:check, test:coverage at 92.44 statements / 89.79 branches / 92.07 functions / 93.89 lines, all over the floors).

Blocking

A .tar.gz restore that fails partway through reports success, and the restore then deletes the user's only other copy.

src/ipc/workers/extraction.ts extractTarGz builds tar.extract with no strict and no warn handler. In node-tar, [ONERROR] (unpack.js) emits 'error' only for CwdError. Every per-entry write failure from [FILE] (ENOSPC, EACCES, EMFILE, an OS-rejected name) goes through this.warn('TAR_ENTRY_ERROR', ...), which, without strict, emits 'warn', then entry.resume() and continues. The extractTarGz promise only listens for unpacker.on("error") (CwdError) and unpacker.on("close", () => finish(unsafeEntry)), where unsafeEntry is set only by the type filter. So a truncated extraction resolves as { ok: true }.

runExtraction then runs validateTree (which only sees what landed) and copyTree, and src/domain/installations/restore.ts moves the original to replacedPath, moves the truncated tree into place, and calls discard(ports, events, replacedPath).

Failure scenario: a user restores a multi-GB backup. runExtraction unpacks into mkdtempSync(join(tmpdir(), "riftlauncher-extract-")). On a /tmp tmpfs sized at half of RAM, it fills partway through. The remaining entries fail with ENOSPC and are warned and skipped. copyTree writes the truncated tree into stagingPath (a sibling of the installation, on a different filesystem, so it succeeds), the swap completes, and the pre-restore installation is deleted.

This is unchanged from base, but base only ran game installs through the tar path, where a truncated result is caught downstream and is re-downloadable. Backup restores went through the zip reader (and before that a 7-Zip child), both of which fail closed. This PR moves every new-format backup restore onto the tar reader, into the one flow that deletes the only other copy.

Fix: pass strict: true to tar.extract, or add unpacker.on("warn", (code) => { if (code === "TAR_ENTRY_ERROR" && !unsafeEntry) unsafeEntry = new Error("Extraction failed") }) so close rejects. Test: extract into a destination made unwritable (or a filter that throws on the Nth entry) and assert runExtraction rejects and no swap happens.

Worth fixing in the same pass, not blocking

  1. extractZip never tears down its streams on failure. src/ipc/workers/extraction.ts: finish() closes zipFile only. On a readStream or writeStream "error" it settles the promise but destroys neither, and runExtraction's finally then runs fse.removeSync(temporaryRoot) over the open write fd. extractTarGz, eleven lines up, handles exactly this with reader.unpipe(); reader.destroy(); unpacker.abort(error) and a comment saying why. On Windows a corrupt deflate entry in a legacy backup gives EBUSY from removeSync, which replaces the real error and leaks the temp folder. Hoist readStream/writeStream into the closure and destroy them in finish.

  2. A failed tar.create leaves a partial .tar.gz on disk. src/ipc/workers/compression.ts: tar.create({ file: archivePath, ... }) opens the file immediately; the catch rethrows without removing it. makeInstallationBackup returns refuse("compress-failed") with no record, so the truncated archive sits in the backups folder invisible to pruneOldestBackups (it walks installation.backups) and accumulates across retries. try { await tar.create(...) } catch { fse.removeSync(archivePath); throw ... }.

  3. assertSafeCompressionTree now returns the byte total but nothing checks it against MAX_ARCHIVE_TOTAL_BYTES. An installation over 2 GiB produces a backup that validateTarGzArchive will always refuse on restore, surfaced only as the generic restore error. Base had the same gap, but the total is now in hand, so refusing in runCompression with a message the backup UI can render is a one-liner.

  4. Legacy zip restores drop unix mode bits on macOS. extractZip uses plain createWriteStream(target) and reads externalFileAttributes only for the symlink check. Linux is covered by the changePerms([outputPath], 0o755) call after every extraction; macOS returns early from that handler. Bounded, since a macOS game version cannot be launched yet, but new .tar.gz backups keep their modes and legacy zips silently do not. Either chmod from the archived mode after the write, or say in the comment that legacy-zip modes are deliberately not preserved.

Minor

  • Progress fidelity. compression.ts onWriteEntry counts a file's full size when its header is emitted, before the body streams, so an installation dominated by one large file jumps to 99 immediately and sits there. The old 7-Zip $progress was byte accurate. Monotonicity, dedup and the single terminal 100 all still hold.
  • tests/fixtures/build-fixtures.ts comment for hostile-backup.zip is wrong. It says the C:/escaped-drive.txt entry "is the one yauzl itself lets through". yauzl's validateFileName refuses /^[a-zA-Z]:/ the same as a leading / or a .. segment, and tests/ipc/extraction.test.ts correctly asserts /could not be read/ for this fixture. All three entries are stopped by yauzl, so validateZipArchive's own isSafeArchiveEntry gate is never exercised by a real archive. Fix the comment; a fixture that actually reaches that gate needs a name yauzl accepts but the launcher refuses.
  • Stale 7-Zip references the sweep missed: src/ipc/handlers/pathsHandlers.ts (the archiveConcurrency rationale is still written around "concurrent 7-Zip processes"; the bound now guards in-process CPU and zlib), src/ipc/pathPolicy.ts ("Every archive the launcher makes is a single .zip file"), and docs/decisions/0001-shell-and-codebase.md (still lists 7zip-bin in the production tree and the six 7za binaries). docs/vintage-story-quirks.md was updated; these were not.

Not checked here

Windows and macOS behaviour (CI runs test on ubuntu only, per #267, so findings 1 and 4 above and the extractZip teardown are all unexercised there), a real timing on a large backup (single-threaded zlib replacing mt=on 7-Zip deflate), and whether any macOS game build is ever published as something other than .tar.gz/.tgz/.zip, which base handled by falling back to 7-Zip and this now hard-refuses.

Pixnop added 2 commits August 29, 2026 16:21
node-tar reports a per-entry write failure as a warning, skips the entry
and closes the stream cleanly, so a restore that filled the disk halfway
through came back looking like a whole one. runExtraction then validated
only what had landed and the restore swapped the truncated tree in and
deleted the copy it replaced.

strict makes those failures errors, so the extraction rejects and the
restore stops before it moves anything. It also covers a corrupt entry
header, dropped just as quietly until now. The two other warnings it
turns fatal are already refused earlier, by the entry type filter and by
the table-of-contents pass.

The zip reader gets the teardown the tar reader already had: a failure
now unpipes and destroys the streams the entry was moving through, so
the temporary folder is not removed out from under an open write handle.
tar opens the archive as soon as it starts, so a write that failed
partway through left a truncated .tar.gz in the backups folder. No
backup record names it, and pruning only walks the records, so it stayed
there for good and every retry added another one beside it.
@Pixnop

Pixnop commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

You are right about the restore path, and thank you for tracing it all the way to the discard(replacedPath) call rather than stopping at the missing handler. Fixed and pushed.

The blocking one

I took strict: true rather than the warn handler, because the handler would upgrade only TAR_ENTRY_ERROR while strict also covers TAR_ENTRY_INVALID, a corrupt entry header whose entry is dropped every bit as quietly and would truncate a restore the same way.

Everything else strict makes fatal is already refused before it can happen, which is the part I wanted to be sure of before picking it. TAR_ENTRY_UNSUPPORTED never fires: the filter runs in the parser, at the point the header is read, so an entry of a type we refuse is ignored and never reaches [UNSUPPORTED]. TAR_ENTRY_INFO for a stripped absolute path cannot be reached either, since validateArchive walks the table of contents and refuses that archive by name before a byte is written. TAR_BAD_ARCHIVE and TAR_ABORT were already errors regardless, Unpack.warn marks them unrecoverable. So the only behaviour that actually changes is the one that had to. Strict has one more thing going for it in the scenario you described: it stops at the first entry that fails instead of running the rest of a multi-GB archive into a disk that has no room left for it.

The test builds an archive whose second entry cannot be written. a is a plain file, and a/b then asks for a to be a folder, so the write fails for that one entry and the unpacker moves on to the next exactly as it does when the disk fills. I went that way instead of an unwritable destination for two reasons. runExtraction unpacks into its own mkdtemp folder, which a test cannot make unwritable from the outside, and a permission based test quietly stops testing anything when it runs as root. Two assertions hang off it: runExtraction rejects and leaves the destination empty, and that same archive driven through restoreInstallationBackup, on a real filesystem with an extractor shaped the way extractWorker.ts shapes it, comes back extract-failed with the installation still holding its file.

Taking strict: true back out fails three tests. The two new ones, plus the existing "writes nothing outside the destination for a tar.gz climbing out" case, which is now stricter than it was. Its .. and absolute entries used to be skipped with the extraction resolving anyway, and they now fail it, so that assertion moved from a plain await to a rejection.

The two smaller ones

extractZip now keeps the entry's read and write streams in the closure and unpipes and destroys them in finish when it settles with a failure, so both ends are stopped before runExtraction's finally gets to the temporary folder. Same shape as extractTarGz eleven lines down. This one has no test, deliberately: the symptom is the Windows EBUSY, and on Linux the removal succeeds whether or not the handle is still open, so a test would pass with or without the fix and say nothing.

A failed tar.create now takes the partial archive away before rethrowing. The test drives a compression that fails partway through, using a source file the safety walk can stat but tar cannot read, and asserts the backups folder is empty afterwards. It carries the same skip guard the account store tests use, since neither Windows nor root would honour the permission. Dropping the removeSync fails it.

Items 3 and 4 and the minor list are untouched here.

Gates

npm ci clean. npm run typecheck, npm run lint:ci at 0 errors and the same 15 pre-existing warnings, and npm run format:check all pass. npm run test:coverage is 137 files, 1646 passing and 2 skipped, at 92.61 statements, 89.87 branches, 92.16 functions and 94.04 lines, every floor clear. Linux only, so the zip teardown stays unexercised for the reason it exists.

Pixnop added 3 commits August 29, 2026 16:32
The reader holds an archive to a 2 GiB total and refuses anything past
it, so an installation over that cap compressed happily into a backup
that could never be put back. The walk already had the total in hand.

Refusing costs the player a failed backup. Not refusing cost them a
backup they would only discover was useless on the day they needed it,
plus a prune slot, since pruning runs before the archive is written and
an older restorable backup had already been deleted to make room.
hostile-backup.zip never reached isSafeArchiveEntry. yauzl's own
validateFileName refuses a drive letter exactly as it refuses a leading
slash or a ".." segment, so all three of its names stop the read before
the launcher's gate has a say, and the comment claiming otherwise was
wrong.

A NUL byte in the middle of a name is one yauzl has nothing to say
about, so unsafe-name-backup.zip is the archive that gets there. Without
it, deleting that check from validateZipArchive failed no test in the
suite.
The concurrency limiter and the worker timeout in pathsHandlers.ts were
written around 7-Zip subprocesses, pathPolicy.ts still said every archive
the launcher makes is a zip, isTarGzName explained itself in terms of
what 7-Zip could not read, and the default compression level was
described as 7-Zip's.

ADR 0001 keeps its measurements. They were taken on a stated day to
support a decision that is still pending, so it gets a dated note saying
the 7-Zip lines no longer hold rather than an edit that would put today's
tree into yesterday's argument.

Also says outright, where the zip reader is, that a legacy backup's unix
modes are deliberately not restored.
@Pixnop

Pixnop commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Second pass, working through the rest of the review. Three of the remaining points are fixed, two are answered rather than changed, and one is fixed differently from the way you suggested. Reasoning for each below.

3. The size the reader will not accept

Fixed, and I think this one was closer to the blocking finding than its position in the list suggests. Refusing an oversized source in runCompression is the substance of it, so the archive is never written.

What makes it worse than "a restore that fails with a generic error" is the ordering in makeInstallationBackup: pruneOldestBackups runs before the archiver, so taking a backup deletes the oldest one to make room and only then writes the new one. An installation over the cap therefore evicted a restorable backup and replaced it with a file that could never be restored, once per attempt, until nothing restorable was left. The archive is now refused before tar opens anything.

Two things I did not do. The prune-before-write ordering itself is untouched: pruning first is what keeps the folder from holding one more archive than the limit at its peak, and changing it is a backup-semantics decision that wants its own change rather than a line in this one. And the message does not reach the UI, because it cannot: compressWorker.ts maps every error to the constant "Compression failed" rather than passing error.message the way extractWorker.ts does, and COMPRESS_ON_PATH returns a boolean, so the renderer sees compress-failed and nothing more. A distinct, renderable reason means a new failure reason in the domain, a locale key in fourteen files and a branch in describeBackupFailure, which is a feature rather than the one-liner, and I would rather propose it separately than smuggle it in here.

The test uses a sparse file: 3 GiB by every stat, no blocks on disk, so it costs nothing and skips on Windows where truncate is not sparse. Removing the check fails it, and takes four seconds doing it, which is the run the guard now avoids.

4. Unix modes on a legacy zip restore

Taken as the second option you offered, documented rather than implemented, and I want to say why rather than just point at the comment.

On Linux nothing preserves modes today, including the tar path: startExtract in TaskManagerContext.tsx runs changePerms([outputPath], 0o755) after every extraction, restores included, which overwrites whatever tar restored. So the asymmetry is macOS only, on a platform where the game cannot be launched, for a folder holding saves and settings with nothing executable in it.

Against that, restoring a mode read out of an archive written by a tool the launcher no longer ships is not free. A zip made on Windows carries 0 in the upper half of externalFileAttributes, and chmodding a save file to 0 would be a worse bug than the one being fixed, so the honest version needs a made-by check, a mask and a zero guard. That is more code, and more risk, than the thing it buys. The comment now says the omission is deliberate and gives these reasons, so the next reader does not have to rediscover them.

Progress fidelity

Not changed, and I think it should stay as it is. The accurate fix is byte accurate, and the only clean way to get there is a counting stream interposed in the pack pipeline, since onWriteEntry fires at header time by construction. Listening on the entry stream instead would count the 512 byte header and the padding as content, which trades one inaccuracy for another and touches the flow of the pipeline that writes the archive. Weighed against a bar that reaches 99 early for an installation dominated by one large save, with monotonicity, dedup and the single terminal 100 all still holding, that is not a trade worth making inside this change. Happy to be overruled if the jump bothers you more than it bothers me.

The fixture comment

You were right, and the fixture was hiding more than a wrong sentence. yauzl's validateFileName refuses /^[a-zA-Z]:/ exactly as it refuses a leading slash and a .. segment, so all three names in hostile-backup.zip stop the read and none of them reaches isSafeArchiveEntry. Comment corrected.

I also took your suggestion for a name yauzl accepts but the launcher refuses. A NUL byte in the middle of a name is one: validateFileName says nothing about it, so unsafe-name-backup.zip reaches the launcher's own gate and is refused there. Worth having, because I checked what the suite did without it: deleting !isSafeArchiveEntry(entry.fileName) from validateZipArchive failed exactly zero of the 1649 other tests. It now fails one.

The rebuild also produced a one byte change in oversized-declared-icon.zip, inside its DEFLATE stream, from a different zlib build rather than from anything in this change. I reverted that file rather than commit the churn.

The stale sweep

Done, plus three the list did not name: the worker timeout comment in pathsHandlers.ts about a task "still holding a 7-Zip child", isTarGzName in validation.ts explaining itself in terms of what 7-Zip cannot read, and DEFAULT_COMPRESSION_LEVEL in defaults.ts described as a 7-Zip level. pathPolicy.ts and the archiveConcurrency rationale are updated as you asked.

ADR 0001 I handled differently, and tell me if you disagree. It is dated, its status is proposed, it says outright that every number in it was measured on 2026-08-16, and Option A's cost paragraph argues partly from the 7za l -slt reader. Editing those lines would put today's tree into an argument that was put to the deciders with the numbers of the day, and would leave every other stale figure in there looking current by association. It gets a dated note instead, saying what no longer holds and pointing at this PR, with the measurements left as they were taken.

Gates

npm run typecheck, npm run lint:ci at 0 errors and the same 15 pre-existing warnings, and npm run format:check all pass. npm run test:coverage is 137 files, 1648 passing and 2 skipped, at 92.70 statements, 89.95 branches, 92.16 functions and 94.13 lines, every floor clear. Both new tests were checked by removing the guard they cover: the size refusal and the name gate each fail exactly one test when taken out.

@Pixnop
Pixnop requested a review from Zaldaryon August 29, 2026 16:16

@Zaldaryon Zaldaryon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Requesting changes on one remaining compatibility issue.

The previous tar extraction failure, zip stream teardown, partial archive cleanup, source size cap, legacy mode documentation, fixture coverage, and stale reference fixes are present in 2346851. The required GitHub checks pass, and I also ran the local RiftLauncher gates: typecheck, lint:ci with 0 errors and 15 existing warnings, format:check, test:coverage with 1648 passed and 2 skipped, and build:unpack.

The new compressor still accepts a source with hard-linked files. assertSafeCompressionTree checks symlinks and special files but not nlink. With tar 7.5.22, tar.create records the second name as a Link entry. The restore validator rejects Link, so the backup operation reports success but the resulting backup cannot be restored. I reproduced this on the PR head with two names for one real inode.

Please reject hard-linked source entries or configure tar to emit independent regular files, and add a regression test showing that a hard-linked source cannot produce a successful unusable backup.

Comment thread src/ipc/workers/compression.ts
Pixnop added a commit that referenced this pull request Aug 31, 2026
* ci(test): run the test job on windows too

Extend the test job to the same os matrix the build job already
uses, so the win32 branches (pathsHandlersWin32, atomic-write
rename semantics, symlink cases that skipIf on win32) run for real
instead of only in their skipped form.

Refs #267

* test: fix windows-only environmental failures the new job surfaced

The first real run of the test job on windows-latest found a handful
of tests that fail purely because of platform differences the tests
never accounted for, not bugs in the code they cover:

- accountLoginFlow.test.ts read the handler source without
  normalizing line endings, so its "\n"-based slice landed in the
  wrong place once git checked the file out with CRLF.
- accountStore.test.ts, configHandlers.test.ts, modsHandlers.test.ts
  and permissions.test.ts all read a POSIX mode bit (0o600, 0o755,
  and friends) back off a real file after chmod. NTFS has no such
  bits; chmod there only toggles the read-only attribute. These now
  skipIf(win32), the same pattern backgroundHandlers.test.ts and
  pathsHandlers.test.ts already use for symlink-only cases.
- pathsHandlers.test.ts had one RUN_INSTALLER test whose own header
  comment already documented it as covering "the not-windows arm,
  real unstubbed behavior on the Linux host these tests run on." On
  an actual windows host that arm can't fire, so it now skips there
  too.

A separate, larger set of gameHandlers.test.ts and extraction.test.ts
failures is left as is; those need a closer look before deciding
whether they're more test gaps or something the launcher itself gets
wrong on Windows.

* test: give the 17 remaining windows failures their verdict

None of them turned out to be a Windows bug in the launcher. The twelve
EXECUTE_GAME failures all came from one fixture assumption: every test
in gameHandlers.test.ts writes a game binary called "Vintagestory", and
buildGameLaunchPlan only ever looks for "Vintagestory.exe" on Windows.
So the folder held no game, the handler answered no-executable, and
every outcome those tests were written for went unreached, adoption
included. Renaming the file by platform in one place restores all
twelve, and the two that looked like they diverged into session
adoption with mismatched uids were the same cascade one level further
down: with no launch there is no session write, so nothing was there to
adopt. Reproduced on Linux by pointing the same helper at a name the
launcher does not know, which fails all twelve with the exact Windows
messages, mismatched uids and all.

Three tests in the same file cannot work on Windows whatever the binary
is called. Two make a write fail by taking write permission off the
installation folder and one makes a folder unlistable with chmod 0o000;
NTFS has no such mode bits, so the write lands and the folder lists.
They skip there with the reason on them.

The three CHANGE_PERMS failures share one cause too: the handler
returns false on anything that is not Linux before it looks at its
arguments, which is right, since POSIX mode bits are the only thing it
has to apply. So the two validation tests get no throw to catch and the
worker test waits for a worker that is never started. Also reproduced
on Linux, by making that early return fire here.

Of the two in extraction.test.ts, one asked the filesystem whether
"vintagestory" exists to prove the wrapping folder was flattened away,
which on a case-insensitive filesystem answers about the "Vintagestory"
file sitting next to it. The full listing on the line above already
says it, and says it better, since an extra folder could not hide from
it either; breaking the flattening still fails the test with that line
gone. The other spends two 7-Zip processes on a 2000 file archive and
runs past the five second default on a Windows runner. Nothing in the
coalescing it covers is platform-specific, and #274 replaces the test
with a yauzl one that spawns nothing, so it skips on Windows for now.

* fix(game): report a spawn that throws as a failed launch, not as an exception

With the fixtures naming the binary Windows actually looks for, the
Windows job got far enough to spawn it, and nine tests then failed on a
raw "spawn UNKNOWN" coming out of the handler itself.

child_process.spawn only reports ENOENT, EACCES, EAGAIN, EMFILE and
ENFILE through an "error" event. Everything else it throws where it
stands, and Windows answers a file that is not a valid executable with
UNKNOWN, which is none of those five. Both spawns in this file were
written for the event alone, so the throw went straight past the
promise and out through the handler. EXECUTE_GAME rejected instead of
resolving launch-failed, which is the exact anti-pattern
gameProcessOutcomeToResult exists to end, and LOOK_FOR_A_GAME_VERSION
rejected instead of reporting no version found. What reaches the player
is a game version whose executable a stopped download truncated or an
antivirus emptied: on Linux that is EACCES and an ordinary "couldn't
run it" notice, on Windows it was the generic error the renderer shows
for an exception, with none of the log lines the failure path writes.

Both spawns now catch it and settle the way the error event does. The
two tests pinning it drive the throw through a spawn wrapper rather
than through a real Windows failure, so they hold the contract on every
platform rather than only where the bug shows.

* ci: keep a job named test so the required context still exists

dev branch protection requires a status context literally named "test", and
a matrixed job cannot produce one: it reports "test (ubuntu-latest)" and
"test (windows-latest)" instead. Rename the matrix job to test-matrix and
add a small gate job that keeps the required name, so the protection rule
needs no coordinated edit.

The gate runs with always() because a plain needs would skip it when a leg
fails, and protection counts a skipped required job as satisfied. It then
compares needs.test-matrix.result against success, which is only the case
when every leg passed, so a failed, cancelled or skipped matrix turns the
gate red.

* fix(game): settle a thrown probe spawn like the error event, and pin line endings

Three follow-ups from review, all in the same file set.

The probe's spawn catch resolved the promise directly while every other
exit from that executor went through settle, because settle closed over a
timer declared below the try and calling it earlier would have hit the
temporal dead zone. The timer now starts as undefined above settle, so the
catch settles like the "error" event does, clearTimeout ignoring an
undefined handle. An asymmetry in how a spawn failure settles is the same
family as the bug this branch fixes, and it was one moved declaration away
from becoming a double-settle.

spawnThrow.next is a vi.hoisted object, which vi.restoreAllMocks does not
touch, so it now resets in beforeEach next to the rest of the per-test
state. No test passes for the wrong reason today: both tests that set the
flag assert it was consumed. The leak needs a test to fail before it
reaches the spawn, and then it lands on whichever test runs next.

.gitattributes normalises text to LF in the repository and on checkout,
which is what a CRLF checkout of accountHandlers.ts needed on the Windows
job. Two other tests read source the same way and pass only because
neither asserts across a line ending. The crafted fixtures are marked
binary so nothing rewrites a byte inside them; tests/fixtures/not-a-zip.bin
is the one that needs it, since it holds no NUL byte and text=auto would
otherwise treat it as text. Nothing in the index is CRLF today, so this
renormalises no existing file.
tar looks a file up by dev:ino whenever its nlink is above one, and on a
hit writes the second name as a Link entry pointing at the first instead
of writing the bytes again. The restore validator refuses Link, so a
source holding two names for one inode produced a backup that reported
success and could never be put back.

Handing tar a link cache that never reports a hit sends every name down
the ordinary file path, so the archive carries both copies and the
restore works. Refusing the source instead would have cost the backup
entirely to players on deduplicating filesystems, which hand out hard
links without anyone asking for one.
@Pixnop

Pixnop commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Went with the second option, so a hard linked source now produces a self contained backup instead of a refusal.

The reason comes down to who ends up holding these. A player does not have to have made a hard link to have one. Deduplicating filesystems hand them out on their own, and so does anything that copies a mod folder with link semantics rather than by reading and rewriting the bytes. Refusing the source would have taken backups away from those installations entirely, and the sharing between two names is a disk layout detail rather than something the installation depends on. Losing it across a restore costs nothing the launcher promises.

tar supports it, though not under a name that says so. In write-entry.js the [FILE] path consults linkCache by dev:ino whenever nlink is above one, and only writes a Link entry on a hit. onWriteEntry fires inside [HEADER], after that decision has already been made, so there is no intervening from there, and 7.5.22 has no noHardlinks flag. What does work is passing a linkCache whose get always answers undefined: every name then goes down the ordinary file path. The deferral in pack.js that holds pending link jobs back is keyed off stat.nlink and released in [JOBDONE], not off the cache, so a cache that never answers does not strand a job.

What the archive holds now, listed off a real one. The source is Mods/carrycapacity.zip with a second name Mods/carrycapacity-1.0.0.zip over the same inode, nlink 2, sixteen bytes of content.

Before: Directory Mods/, File Mods/carrycapacity-1.0.0.zip size 16, Link Mods/carrycapacity.zip size 0 linkpath Mods/carrycapacity-1.0.0.zip

After: Directory Mods/, File Mods/carrycapacity-1.0.0.zip size 16, File Mods/carrycapacity.zip size 16

Both names carry their own bytes. After a restore the two files have nlink 1 and different inodes, which is also what validateTree wants: it refuses a hard link in an extracted tree, so putting the sharing back would have been refused at the far end regardless. One side effect worth naming: assertSafeCompressionTree already counted both names toward the total, and the archive now genuinely holds both, so the size cap and the progress figure describe what actually gets written rather than overstating it.

The regression test sits in the backup round trip block in tests/ipc/extraction.test.ts. It builds the pair with fs.link, asserts the two names really do share an inode before anything else happens, compresses, lists the archive and asserts that no entry is anything other than File or Directory and that both mod names carry their sixteen bytes, then restores and compares the tree byte for byte against the original and checks the two restored files are separate inodes.

Mutation, with the linkCache line taken back out: the compression still resolves, the archive comes back holding the Link entry above, and runExtraction on that same archive rejects. So the state being guarded against really is a backup that reported success and cannot be put back, not merely a different archive layout. The regression test fails on the entry type assertion in that state and passes again with the line restored.

Gates on Linux with Node 22: typecheck clean, lint:ci at 0 errors and the same 15 existing warnings, format:check clean, test:coverage at 137 files, 1649 passed and 2 skipped, 92.7 statements, 89.95 branches, 92.17 functions, 94.13 lines, every floor clear.

Not checked here: Windows, where NTFS has hard links too but the suite does not run.

@Zaldaryon Zaldaryon left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after re-review of the updated range.

The hard-link fix in d7e6fe9 now passes tar a link cache whose get method never reports a previous inode. A source with two names for one inode is written as two independent regular files, so the resulting backup remains acceptable to the restore validator. The regression test covers the real inode, archive entry types and sizes, restore contents, and separate output inodes.

The PR was conflicting with the current dev branch because the base gained eight commits after the previous review. Merge commit c3471c5 resolves the conflict in tests/ipc/extraction.test.ts while retaining the current temporary-directory isolation test and the backup round-trip coverage.

Local verification on the integrated head passed: 97 targeted archive and backup tests with 1 skip, npm run typecheck, npm run lint:ci with 0 errors and 15 existing warnings, npm run format:check, npm run test:coverage with 1678 passed and 2 skipped, and npm run build:unpack.

GitHub typecheck, lint, Ubuntu and Windows test matrix, SonarCloud, and Ubuntu and Windows builds all pass. The macOS build is skipped by workflow policy. The hard-link thread is resolved and no blocking findings remain.

@Pixnop
Pixnop merged commit 3296c1d into dev Aug 31, 2026
9 checks passed
@Pixnop
Pixnop deleted the refactor/drop-seven-zip branch August 31, 2026 15:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants